Categorical Foundations of Gradient-Based Learning

We propose a categorical semantics of gradient-based machine learning algorithms in terms of lenses, parametrised maps, and reverse derivative categories. This foundation provides a powerful explanatory and unifying framework: it encompasses a variety of gradient descent algorithms such as ADAM, AdaGrad, and Nesterov momentum, as well as a variety of loss functions such as as MSE and Softmax cross-entropy, shedding new light on their similarities and differences. Our approach to gradient-based learning has examples generalising beyond the familiar continuous domains (modelled in categories of smooth maps) and can be realized in the discrete setting of boolean circuits. Finally, we demonstrate the practical significance of our framework with an implementation in Python.Comment: 14 page

A Genetic Algorithm for Cost-Aware Business Processes Execution in the Cloud

International audienceWith the generalization of the Cloud, software providers can distribute their software as a service without investing in large infrastructure. However, without an effective resource allocation method, their operation cost can grow quickly, hindering the profitability of the service. This is the case for BPM as a Service providers that want to handle hundreds of customers with a given quality of service. Since there are variations in the needed load and in the number of users of the service , the allocation and scheduling methods must be able to adjust the cloud resource quantity and size, and the distribution of customers on these resources. In this paper, we present a cost optimization model and an heuristic based on genetic algorithms to adjust resource allocation to the needs of a set of customers with varying BPM task throughput. Ex-perimentations using realistic customer loads and cloud resources capacities show the gain of these methods compared to previous approaches. Results show that, in our case, using our algorithm on split groups of customers can provide better results

Neighbourhood analysis: A case study on Google machine reassignment problem

It is known that neighbourhood structures affect search performance. In this study we analyse a series of neighbourhood structures to facilitate the search. The well known steepest descent (SD) local search algorithm is used in this study as it is parameter free. The search problem used is the Google Machine Reassignment Problem (GMRP). GMRP is a recent real world problem proposed at ROADEF/EURO challenge 2012 competition. It consists in reassigning a set of services into a set of machines for which the aim is to improve the machine usage while satisfying numerous constraints. In this paper, the effectiveness of three neighbourhood structures and their combinations are evaluated on GMRP instances, which are very diverse in terms of number of processes, resources and machines. The results show that neighbourhood structure does have impact on search performance. A combined neighbourhood structures with SD can achieve results better than SD with single neighbourhood structure

Parallel Late Acceptance Hill-Climbing Algorithm for the Google Machine Reassignment Problem

Google Machine Reassignment Problem (GMRP) is an optimisation problem proposed at ROADEF/EURO challenge 2012. The task of GMRP is to allocate cloud computing resources by reassigning a set of services to a set of machines while not violating any constraints. We propose an evolutionary parallel late acceptance hill-climbing algorithm (P-LAHC) for GMRP in this study. The aim is to improve the efficiency of search by escaping local optima. Our P-LAHC method involves multiple search processes. It utilises a population of solutions instead of a single solution. Each solution corresponds to one LAHC process. These processes work in parallel to improve the overall search outcome. These LAHC processes start with different initial individuals and follow distinct search paths. That reduces the chance of falling into a same local optima. In addition, mutation operators will apply when the search becomes stagnated for a certain period of time. This further reduces the chance of being trapped by a local optima. Our results on GMRP instances show that the proposed P-LAHC performed better than single threaded LAHC. Furthermore P-LAHC can outperform or at least be comparable to the state-of-the-art methods from the literature, indicating that P-LAHC is an effective search algorithm